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Abstract 
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Sensitivity analysis plays an important role in the understanding of complex 



£SJ ' models. It helps to identify the influence of input parameters in relation to the 

ly-v , outputs. It can also be a tool to understand the behavior of the model and 

m 
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lustrate the potential usefulness of combining first and second-order sensitivity 
>—* | analysis, applied to a building energy model (ESP-r). Through the example of 

an apartment building, a sensitivity analysis is performed using the method of 
elementary effects (also known as the Morris method), including an analysis of 
the interactions between the input parameters (second-order analysis) . The use- 
fulness of higher-order analysis is highlighted to support the results of the first- 
order analysis better. Several aspects are tackled to implement the multi-order 
sensitivity analysis efficiently: interval size of the variables, the management of 
non-linearity and the usefulness of various outputs. 
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1. Introduction 

Energy consumption related to the building sector is recognized as a major part 
of the total energy consumption worldwide (37% of the final energy consumption 
in the EU in 2004) [l| and consequently a significant source of greenhouse gas 
emissions [2j|. The growth in population, building services and comfort levels 
guarantees that this tendency will continue in the forthcoming years. Many tools 
have been developed to model the energy consumption in buildings (EnergyPlus, 
TRNSYS, ESP-r), particularly for end uses such as space heating and cooling, 
ventilation and lighting. In most cases, such models take into account coupling 
between phenomena (e.g. interactions between occupancy, micro-climate, enve- 
lope, and HVAC.) through coupling of different specialized sub-models and by 
using a large number of diverse input variables. 

Sensitivity analysis can help in understanding the relative influence of input 
parameters on the output [3]. In the field of building energy models, combining 
sensitivity analysis and simulations tools can be useful as it helps to rank the in- 
put parameters (or family of parameters) and then to select the most appropriate 
to be considered, depending on the objective of the modeling. For example, this 
is particularly interesting when the modeling objective is related to the building 
design (e.g. sketch stage of the design, modeling retrofitting scenarios according 
to the only available input data) or when it is to define archetypes. Another 
application is in the development stage of the tools, and more precisely the defi- 
nition of possible simplification of models or in the validation of assumptions in 
the selection of input parameters that must be considered. In these cases, and 
depending on the objectives of the tool developed, some sub-models and their 
corresponding input data may become secondary. A solution consists of using 
a detailed model in the upstream stage, combined with a sensitivity analysis 
in order to rank the set of parameters and identify the coupling between them. 
Then, the selection of the most important variable helps to define the structure 
of the simplified model. 
In this study, we propose combining the implementation of ESP-r [4] with two 
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sensitivity analysis techniques: the Morris method |5j and an extension of this 
methodology for the analysis of interactions between the parameters [6|. In 
section 2, sensitivity analysis methods are quickly reviewed. Then, the elemen- 
tary effects method and its second-order variant are described, together with 
the apartment building test case (section 3). Finally, results are presented and 
discussed for the two methods used (section 4). 

2. Background 

2.1. Sensitivity analysis : current approaches 

Sensitivity analysis methods have been studied by many authors in the past 
decades as they have demonstrated their strength in many sectors. Through- 
out this period new methods and improvements have been developed, offering 
different solutions depending on the objective. Hamby [7] proposed an inven- 
tory of techniques for parameter sensitivity analysis which he divided into three 
different categories: 

- Sensitivity analysis methods assessing the influence of individual parameters. 
These include Differential Sensitivity Analysis, One-at-a time sensitivity mea- 
sures, Factorial Design, Sensitivity Index, Importance Factors, and Subjective 
Sensitivity Analysis. 

- Parameter sensitivity analysis utilizing random sampling methods (simple ran- 
dom sampling, Monte Carlo, Latin Hypercube). In this group are listed the 
methods: Scatter plots, Importance Index, 'Relative Deviation', 'Relative Devi- 
ation Ratio', Pearson's r, Rank Transformation, Spearman's p, Partial Corre- 
lation Coefficient, Regression, and Standardized Regression techniques. 

- Sensitivity tests involving segmented input distributions: the Smirnov test, the 
Cramer Von-Mises test, the Mann- Whitney test, and the squared-ranked test. 
The author then applied these different methods to a case study related to the 
nuclear industry, in order to compare them in terms of reliability, computational 
requirements and ease of implementation [8]. The study identified the One-at- 
a time method as being the simplest but pointed out that it becomes time- 
intensive with large numbers of parameters. Saltelli et al. [9] also describe the 



different sensitivity analysis techniques. For these authors, techniques can be 
divided between global and local methods. Local methods are commonly based 
on the estimation of partial derivatives in order to obtain a qualitative analysis 
of the importance of each factor on the output response for a limited subset and 
particular values of the input variables. Global methods vary all the parameters 
and try to obtain information for a subset of input variables in a wider domain. 
Global methods can also be divided into quantitative and qualitative techniques. 
Santner et al. [10] developed the parallel between the physical experiments and 
the concept of computational experiments as it is understood in this study. In 
particular, these authors described the added value of sensitivity analysis in 
such an experiment. This was taken up by Saltelli et al. [9j who presented the 
One-at-a time sampling for sensitivity analysis for multiple parameters. 

2.2. Principles of the elementary effects method 

The Morris method is derived from OAT (One-factor-at-a-time) screening meth- 
ods to identify the subset of the main important input factors among a large 
number of k input parameters in a model. This method characterizes the sen- 
sitivity of a model with respect to its input variables through the concept of 
elementary effects, which are approximations of the first order partial deriva- 
tives of the model |5| . These elementary effects are estimated at various sampled 
points, randomly selected on a p- values regular grid, defining a relevant design of 
computational experiments. The average and standard deviations of elementary 
effects enable negligible and influencing variables to be sorted and linear and 
non-linear influences to be distinguished. In some respect, this method can be 
considered intermediate between a local sensitivity analysis and global quanti- 
tative methods described above. It is a general approach (model-independent), 
which achieves a good compromise between accuracy and efficiency. Applica- 
tions can be found in a number of fields including Environmental Modeling and 
Agriculture [11], Biophysics [12] and Nuclear Engineering [13). However, in spite 
of its advantages, its applications still remain limited. 
Other methods, such as variance-based sensitivity indices (VBM), have been 



proposed [14( . Although they generally provide better information to distinguish 
non-linearities and interactions [15}, the computational cost is much higher: a 
variance-based analysis for a 12-input parameter model requires at least 14,000 
runs of the model, about one hundred times the cost of a first-order Morris 
analysis (and still ten times more than a second-order Morris analysis). 

2.3. Experience with the elementary effects methods for building thermal simu- 
lation 

Some studies have tested the advantages of the Morris method applied to build- 
ing energy simulations. Breesch and Janssens |16| implemented it to identify 
the most important parameters that cause uncertainty in the predicted perfor- 
mances of natural night ventilation. They used a two-zone model in the thermal 
simulation tool TRNSYS coupled with an infiltration model COMIS. This anal- 
ysis revealed that the internal heat gains, local outdoor temperature and the 
diurnal internal convective heat transfer coefficient were the parameters with 
the greatest impact on thermal comfort. Brohus et al. [17[ applied the method- 
ology to reduce a set of 75 parameters used to obtain an accurate output energy 
consumption distribution. The Morris method was also used as a first indica- 
tion of correlation or non-linear effects between the parameters. Finally, this 
method was compared with the Fourier Amplitude Sensitivity Testing method 
(FAST) . The results of both analysis helped in evaluatinga safety factor for the 



annual energy consumption at the design level. De Witt [18( compared the Mor- 
ris method with the sequential bifurcation technique using a mono-zone office 
of 81 parameters as a model. Both techniques found the same set of important 
parameters (12) which explained 94% of the variability of the model output de- 
fined as the number of hours of overheating. Corrado and Mechri [19fl analyzed 
the heating and cooling needs of a two-storey single-family house in Turin with 
the Morris method to calculate the uncertainties in energy rating. The sensi- 
tivity analysis showed that only 5 of 129 factors were responsible for most of 
these uncertainties: the indoor temperature, the air change rate, the number of 
occupants, the metabolism rate and the equipment heat gains. Heiselberg et al. 
[20J identified the most important design parameters in relation to a building's 



performance with a focus on the optimization of sustainable buildings. They 
found that the mechanical ventilation rate in winter and lighting control were 
the most influential parameters in an office building of 7 floors. 
The extension of the Morris method for second- and upper-order analysis has 
still not been applied in the area of building energy simulations despite its 
advantages stated in other analysis and its low computational cost compared to 
more sophisticated techniques like variance-based and FAST methods. 

3. Methodology 

3.1. The elementary effects method 

The building thermal model can be represented by a function y(x) where y is the 
output variable of interest (scalar) and a; is a vector of real input variables with 
k coordinates, each input variable being defined within the range of a continuous 
interval. Input variables are transformed into reduced dimensionless variables 
in the interval (0; 1) as x', ; — z, ~ :r """ . 

Xmin and x max are the minimum and maximum of the input variable x% , respec- 
tively. The domain of the vector x is then a hypercube H k with unit length, 
a subset of M. k . For each reduced input variable, only discretized values are 
considered, using a pi values regular grid (with step _. ): 0; _. ; _ •...; 1. 
Although a single grid value is generally used for all the variables, it is possible 
to use a specific one for each input variable Xi. This enables qualitative input 
variables with two levels to be incorporated, represented in the Morris design by 
a 2-value grid, while keeping a more precise grid for continuous input variables. 
A simulation trajectory is defined as a sequence of (k + 1) points in this hyper- 
cube, with each point differing from the preceding one by only one coordinate. 
In a trajectory, each input parameter only changes once with pre-defined step 
Ai. The function y (i.e. the simulation model) is evaluated for every point in 
the trajectory. The first point of a trajectory is randomly selected. At each step 
in a trajectory, the coordinate to be modified is also randomly selected. Thus 
various trajectories differ by their starting point and by the order of modified 
coordinates. In order to ensure a strict equal probability for each value, Morris 



suggested taking p even and Aj = 2 , p _^ . So, if the initial value x% is lower 
than 0.5, the final value is Xj + A,-; if it is greater than 0.5, the final value is 
Xi — Aj. Taking this Aj value corresponds to a uniform distribution over the 
discrete grid values in intervals [0;1] for all input variables over the hypercube 
H k . 

A trajectory enables a coefficient of variation for each input variable i, called an 
elementary effect (EE) to be evaluated. It is computed between the two points 
of the trajectory where this input variable i is modified (equation [T]) 

EE, = (1) 

ei is a vector of zeros but with its z-th component equal to ±1. Each trajectory, 
with its (fc + 1) simulations, provides an estimate of the k elementary effects. A 
set of r different random trajectories (with index t) is defined in the hypercube 
of input variables; it provides r estimates EEn of elementary effects related to 
each input variable i, at the cost of r x (fc + I) simulations. The average and 
the standard deviation of the elementary effects are computed for each input 
variable i (equations [5] and [3]) : 
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Campolongo et al. [21] recommend using the average of absolute elementary 
effects (equation@l rather than the usual average, since some elementary effects 
can eliminate each other in non-monotonic models. 
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The criterion /i* is a good indicator to classify input variables by order of impor- 
tance, despite the fact that information about the sign of the elementary effects 
is lost. Moreover, the standard deviation of the elementary effects is a relevant 



indicator of non-linearity in input parameters of the model or interactions with 
other parameters involved in the model. By plotting both statistical indicators, 
the Morris method identifies the inputs that can be considered to have an effect 

1. Negligible (low average, low standard deviation) 

2. Linear and additive (high average, low standard deviation) 

3. Non-linear or involved in interactions with other input parameters (high 
standard deviation). 

3.2. The second-order interactions sensitivity analysis 

If parameters show a marked non-linear behavior and a significant influence in 
the model, a subsequent experiment with only these inputs is recommended. 

i 



Campolongo and Braddock 



22j proposed an extension of the Morris method 
which enables the calculation of at least second-order effects with a reasonable 
computational cost. This method is based on the calculation of the equivalent 
of a second-order derivative of the model and optimizes the computational ex- 
periment by using the solution of the "handcuffed prisoner problem" [23]. Thus, 
the number of evaluations of the model required to obtain one second-order ele- 
mentary effect is optimized (about k 2 r). Campolongo and Braddock [6] defined 
a second-order elementary effect (equation [5]) . 

EEj EE. 
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where SEE^ characterizes the influence due to the change of both factors i and 
j at the same trajectory (equation [5]) 

SEEi * ~ A^ (6) 

A further extension to the analysis of third-order (or higher) interactions was 
proposed by Campolongo and Braddock [6J. Unfortunately the computational 
cost of this analysis can be prohibitive if the model is too complex. One practical 
solution is to perform a second-order experiment first and then an analysis of 



third-order effects only in the group with the highest values of standard deviation 
in the second-order analysis. 

Statistics can be applied to second-order elementary effects so that the results 
of the mean, absolute mean and standard deviation of the effects can be inter- 
preted in the same way as in the first-order Morris method [6|. In addition, 
as the standard-deviation of the first-order provides information about second- 
and higher-order interactions between the parameters, the second-order exten- 
sion can give valuable data about the interactions of third- and upper-order 
degrees. The combination of both methods at the same intervals enables the 
importance of a parameter in the first, second, and higher orders to be classified. 
This combination gives a better interpretation of complex models and can be 
applied to the construction of reduced models based only on the most important 
parameters. 

3.3. Test case: an apartment building 

In order to explore the potential of the elementary effects method, a rather com- 
plex multi-zone building was chosen as a test case. It is a seven-storey residential 
building with 32 dwellings and an estimated population of between 70 and 80 
inhabitants (Figure [I]). It has an approximate floor area of 3500 m 2 , and the 
average yearly energy needs for heating are 67.59 kWh/m 2 . The characteristics 
of this building are given in Table Q] It is connected to a heating network and 
no cooling equipment is available. The glazing ratio is 30%, composed mainly 
of double glazed windows. All exterior walls are insulated with 10 cm of insu- 
lating material. The building is divided into 24 thermal zones (two heated and 
non-heated zones per floor). 

The building was modeled using the energy calculation software ESP-r [4| . The 
choice of ESP-r was driven by its detailed sub-models as well as the numerous 
input variables considered in this tool, allowing many possible situations of in- 
fluential factors to be investigated. The p-dimensional grid was defined as p=10 
for all experiments, so each reduced input variable could take the discretized 
values 0, 0.111, 0.222, ...,0.889, 1 in the reduced interval [0; 1] and simulations 



were made with regularly spaced values between a minimum and a maximum. 
The numerical experiments were classified into two different sets (A, B) based on 
the number of parameters involved in the analysis (see Table [5] for the definition 
of all input variables and their range) . 

In the set of experiments (A), a first-order effects sensitivity analysis was per- 
formed with 24 parameters, representing different choices in building design. 
These design parameters were the corrections for building height (input 1), 
width (2) and length (3) as well as the building rotation (24), insulation thick- 
ness (23) and glazing ratio (12-15). Corrections of the weather parameters used 
in the analysis were added in a first attempt to quantify the importance of the 
environment of the building in the different outputs studied (19-22). 
For some parameters, common values for all heated zones except one were taken: 
heat gains due to occupants (8), the set point temperature (4), the ventilation 
rate (10) and the difference in the set point temperature between day and night 
(6). The objective was to see the influence of these common parameters com- 
pared to the climatic and design factors that also affect the entire building. 
In addition, to determine the relative influence of these parameters in a single 
thermal zone, apartment E42 (which is located on the fourth floor) was chosen 
arbitrarily to change these parameters only in this specific zone. However, the 
specific results from this apartment are not presented here, as they do not affect 
the results for the building as a whole (parameters 5-7-9-11). 
For the first numerical experiment (A), rather broad intervals were chosen for 
all input variables, representing the diversity of the characteristics of apartment 
buildings in an urban context on a large scale (e.g. at national level), regarding 
size, insulation, glazing ratio, and ventilation. In a similar way, broad ranges 
were taken for variables concerning the building environment (particularly the 
climate) and occupancy. Nevertheless, the values adopted remain representative 
of the variations or uncertainties usually considered. It was a deliberate choice 
to start with such a "maximum- variability" scenario in order to be representative 
of apartment buildings at a national level. One possible application could be to 
identify the data to be collected with high priority for a good description of the 
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building stock in a region or a country. 

In the second-order Morris extension experiments (set B), only 12 of 24 param- 
eters from the previous experiment were selected. This choice was made due to 
the high computational cost of the analysis. 

4. Results and discussion 

4-1. General remarks on the presentation of results 

The results of the elementary effects analysis of building thermal simulations 
are presented as scatter plots with a point for each input variable i: the x-axis 
represents the absolute average (/Xj*) and the y-axis represents the standard- 
deviation of the elementary effects (<jj) (see, for example, Figures [3] to [5]) . 
The absolute average (/x*) was introduced above as a measure of importance for 
the input factor i. This information can be complemented by the ratio (o~i//J,*) 
as an indicator of linearity (or non-linearity), as justified below. 
First, looking at one input variable i : if all estimates of elementary effects EEu 
have the same sign, we can say that this input factor i has a monotonic effect on 
the response y, increasing or decreasing, depending on the sign of the elementary 
effects. In this case, /x* is equal to the absolute value of jm- The reverse is also 
true : if /x* = abs (/Xj), the effects of input variable i are monotonic. 
Using well-known statistical properties, if elementary effects are assumed to be 
normally distributed, 95% of EE-esthaa,tes are within the range (/Xj ± 1.96 0$). 
As a consequence, if cr^/ix.; is smaller than 0.10, most elementary effects (95%) 
are in a range ±20% around /x^ the elementary effects are almost constant and 
the input variable i has an almost linear effect on the model. A true linear 
response correspond to Oij '/Xj = 0. 

If the ratio Oij [ii is smaller than 0.5, most elementary effects (95% with the 
normal assumption) have the same sign and the model response can be consid- 
ered as monotonic with respect to the input factor i. Thus, in this case, it can 
be considered that /j,i » abs(pi) and <7j//x* « Oi/abs (/Xj). This fact justifies 
using the ratio Ui/jJ,* as an indicator for almost linear (if <0.1) or monotonic 
influences (if < 0.5). 
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As the distribution of elementary effects may be far from normal, another justi- 
fication is presented in Figure[5]with a scatter plot of (ai/fi*) vs. ai/abs (/ii) for 
all first-order analyzes performed within the present work (sets A and Bl, B2). 
This diagram shows that for ai/abs (/i^) smaller than one, most of the points 
are on the bisector (ci//i*) = ai/abs (fii), indicating a monotonic (or almost 
monotonic) behavior in a wider interval than expected (not only < 0.5). For 
highly scattered elementary effects (ai/abs (fii) > 1), a non-monotonic behavior 
is clearly established and, in this case, (ci//z*) stays in the interval between 1 
and 2. The absolute average /i* is very different from the average abs (/Xj) and 
is more influenced by the standard deviation ai. 

So, by plotting three straight lines of slopes a / [i* =0.1, 0.5 and 1, respectively, 
we can graphically identify in the elementary effects scatter plot, those factors 
which are almost linear (below the line a / [i* — 0.1), monotonic (0.5 > a/ \i*> 
0.1) or almost monotonic (1 > a / /i* > 0.5), and those factors with marked non- 
monotonic non-linearities or interactions with other factors (a / '/i* > 1). 
Defining these four zones also provides a means of checking the results of the 
sensitivity analysis if the results contradict what is understood from the physical 
point of view. 

For some of the figures (for example Figs. [5cll5d[l5cl and [6d| . a slightly different 
presentation of data has been used with the ratio Ci//i* on the vertical axis 
(the different domains defined above as linear, monotonic and highly non-linear 
being distinguished by horizontal lines). This presentation provides exactly the 
same information but is sometimes easier to read as the different points are 
better separated. 

4-2. Computational experiment A: first- order analysis results 

For the first set of numerical experiments (set A), several characteristics for 
output results were examined using the first-order elementary effects analysis: 
yearly heating loads (Fig. I5aj) . outputs derived from the latter (heating load 
per m 3 ; logarithmic transformation) (Fig. I3bl - [3d | . heating power (with the 
example of the power exceeded during 1000 hours/year) (Fig. I4bl) and the 



12 



summer comfort factor, using the average internal temperature in July-August 

(Fig. Ba). 

Before presenting the results, it is important to recall the definition of elemen- 
tary effects: one value of the elementary effect for the input variable i corre- 
sponds to the output variation when the input i moves from the minimum (0) 
to the maximum (1). For example, in Figure [5al the average elementary effect 
for the building height (input 1) has a value of 470 000 kWh/year. This is the 
average influence of a modification of building height from 9 m to 27 m (nom- 
inal height is 18 m, corrected by ±50%). For the building considered and the 
wide intervals taken for input parameters, the yearly energy needs are within 
the range of 57 587 kWh/year to 788 894 kWh/year. 

The first analysis based on the yearly heating load (Fig. I3"af highlights the 
dimensional parameters (numbered 1, 2 and 3), indicating a typical size effect: 
building size is the major influence on heating demand. Next, a second group of 
input parameters appears with a significant influence: set point temperature (4), 
ventilation rate (10) and insulation thickness (23). The remaining parameters 
can be classified as less important, although not negligible for a number of them 
(6, 8, 12, 13, 19, 20, 21, 24) which will be discussed below in this section. 
Only one parameter has an almost linear effect (occupant free heat gains, 8) 
with a /(i* close to 0.1. All other parameters show a non-linear influence and/or 
interactions with other parameters (ct//i* > 0.5 for many of them). Nevertheless, 
they generally stay within the monotonic zone (er///*< 1) with the notable 
exception of insulation thickness (23). 

An attempt to eliminate the size effect is made through the analysis of annual 
heating load per cubic meter, i.e. load divided by the three-dimensional param- 
eters (Fig. [3c]). The second group of parameters identified in the former case 
(4, 10 and 23) now becomes predominant, with a behavior closer to linear for 
the set point temperature (4) and ventilation rate (10) (<r//x* equals 0.3 and 
0.2, respectively). 

A third group of parameters is emphasized by this presentation, including factors 
depending on occupants (free heat gains, 8, and heating set point night reduc- 
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tion, 6), solar gains (diffuse, f9, and direct, 2f, radiation potential; building 
rotation, 24; glazing ratios on main facades, f 2 and 13) and climatic sensitivity 
related to the building environment (external temperature, 20) . Compared with 
Fig [3a] all these parameters coincide with those already identified as secondary 
but non-negligible. 

The importance of dimensional parameters (f , 2 and 3) is clearly reduced, but 
they remain among the non-negligible variables and contribute to the high vari- 
ability of the model (a /[/,* close to 1). 

A logarithmic transformation is applied to the output to identify interactions 
caused by possible multiplicative effects (Figure I3bl for annual heating load and 
I3dl for heating load per cubic meter). Logarithmic transformation is often used 
for the statistical analysis of positive outputs with variations of several orders 
of magnitude or when multiplicative phenomena take place. If the response 
function can be expressed as a multiplicative function of various inputs xi, 22, 
23... as y{x) — fi(xi) x ^2(^2) X /3(^3)..., the elementary effects of y vs. in- 
put variables x±, X2, X3... are highly influenced by interactions between Xi, 
X2, X3... The standard deviations of their respective elementary effects can be 
expected to have high values. Taking the logarithm of y(x) separates the terms 
ln(y(x)) = ln(fi) + ln(f%) + ln(fz)... and eliminates interactions: for the loga- 
rithm output, the standard deviation of EE depends only on the curvature of 
function ln(fx);ln(f2)',ln(f3)-» Another advantage of the natural logarithmic 
transformation is the interpretation of the elementary effects as rates of change 
of the output when the value remains low (typically <0.5). For instance, regard- 
ing the input variable 'night and day difference temperature' (6), the average 
elementary effect (expressed in logarithm) is 0.35 (here a decrease, i.e. -0.35, 
Figure [3d)) . This means that a full-scale change in this input (from K to 8 K) 
leads to a 35% variation in space heating, on average. 

The logarithmic transformation provides consistent results with the former ones: 
the same input parameters are identified as important and non-important with 
about the same ranking and similar clusters. The use of a logarithmic trans- 
formation emphasizes some factors with partial multiplicative effects, mainly 
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temperature-related parameters: set point (4), night reduction (6) and exter- 
nal temperature (20). Their a/fi* ratios are both reduced significantly when 
compared with Fig. I3"al and [3bl and are close to 0.1. 

Other parameters remain non-linear with the logarithmic transformation, such 
as parameters related to the glazing ratio (12, 13) and rotation (24) or insulation 
thickness (23). It can be noted that the two presentations (Fig. I3bl and [3d]) in 
the logarithmic transformation lead to exactly the same coordinates for all input 
parameters, except the dimensional ones (1, 2 and 3). 

As a partial conclusion, all the presentations show consistent results in terms 
of predominant parameters. The high variability of almost all the parameters 
encourages the use of a methodology to analyze the effects of interactions and 
non-linearities. It is notable that the most important parameters in the analysis 
(ventilation and set point temperature) are in accordance with a similar analysis 
carried out by Brohus et al. [l_7J for a single-family detached house model with 71 
input parameters. The glazing ratio appears to be less important for the model 
output of energy demand for heating. Similar conclusions were also reported 
by Gasparella et al. [2J| who did an analysis and modeling of various types of 
glazing in a high insulated building with a high percentage of double glazing. 
The Morris method analysis was applied to the average internal temperature 
in July and August (Figure I4a[) and to the power in the monotonic curve at 
hour 1000 (Figure I4b[) . The objective was to show the applicability of this 
method to different kinds of output. By comparing Figures I3bl and I4b[ we 
can conclude that there is a strong relationship between the output of heating 
needs in ln(kWh/year) and the monotonic curve of power. Only the set point 
temperature difference between night and day (6) becomes less important in 
terms of average elementary effects but with a higher standard deviation. On 
the contrary, the comfort criterion ( Figure [4"a|) shows a very different behavior. 
The most important parameter is related to the external temperature with a 
highly linear effect. The next important parameters (with a high fj,i*) are related 
to the environment or the geometry of the building. According to Figure I4al 
the glazing ratio of side B (parameter 13) seems to have more importance in 
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the variability of the average temperature than the other sides (parameters 12, 
14 and 15). This implies that parameter 13 could have a major impact on the 
comfort level of the building. The same conclusions can be made by analyzing 
the geometrical parameters (1, 2 and 3). Thus, the Morris method could also 
serve as a diagnostic tool to help designers to review the impact of the building 
on comfort levels and to analyze in more depth the sensitivity of the building 
in terms of solar gains. 

4-3. Computational experiment B: first- and second-order results 
4-3.1. Data used in experiment B 

In order to make the analysis done with the first-order approach in experiment 
A more precise, experiments Bl to B4 were performed, focusing on the study 
of the second-order interactions. Due to the high computational cost of second 
order analysis (5760 runs with r = 10), a subset of only 12 from 24 initial input 
parameters was selected (see Tabled- Parameters with negligible influence were 
omitted and important parameters (from experiment A) were kept with excep- 
tions for some parameters playing similar roles: among size-related parameters 
(1, 2, 3), the height was omitted and among the glazing ratio of the facades (11, 
12, 13, 14), only the one for facade A (11) was kept. Furthermore, the analysis 
was done for two interval sizes for the parameters: a large one (the same as the 
one used previously for the first-order experiment) and a small one (Table 2). 
These small intervals may be considered representative of the level of knowledge 
or uncertainties of the main parameters in the situation of a detailed energy au- 
dit (with measurements being made inside the building). Small intervals can 
be considered for parameters known approximately while large intervals can be 
used for totally unknown parameters. 

Two different outputs are presented: the annual heating needs and the natu- 
ral logarithm of the annual heating needs per cubic meter. Before running the 
second-order experiment, a first-order analysis is made, limited to the 12 input 
variables selected. So experiment B actually incorporates 4 numerical experi- 
ments: Bl (1st order, large interval, same interval as experiment A), B2 (2nd 

16 



order, large interval), B3 (1st order, small interval) and B4 (2nd order, small 
interval) . 

4-3.2. First order - large interval (Bl) 

A comparison between the two first-order experiments Bl (12 parameters: Fig. 
I5al and I5b[) and A (24 parameters: Fig. [3a] and I3d[) demonstrates that the 
decrease or increase in the number of parameters involved in the sensitivity 
analysis does not significantly affect the ranking between the parameters. When 
comparing the two sets of results, one should keep in mind that differences may 
come from two sources: 

- the Morris Method is basically a Monte Carlo approach with random genera- 
tion, so changes may come from sampling differences, 

- reducing the number of variables (with unchanged intervals) results in fewer 
interactions between input variables, so a lower standard deviation for elemen- 
tary effects can be expected. This trend is actually observed for most of the 
variables for the two output responses considered, but with a limited decrease 
in the standard deviation. 

4.3.3. Second order (B2) 

For the second-order analysis (experiment B2, Fig. l5cl and [5df . it can be seen 
by comparing the x-axis of Figures [5a| and [5c] that the second-order influences 
are not negligible. Figure [Sc] (with logarithmic response) shows that the average 
of the second-order elementary effect of the highest influence interaction (3;8) is 
almost 0.2, exceeding some significant first-order influences of parameters (e.g. 
2 and 24) (Figure [S"a[). This influence becomes even more evident in the analysis 
of annual heating needs in kWh (Figures I5bl and [5d]) . The average second-order 
elementary effect of the highest influence (3,2) exceeds first-order averages of 
elementary effects in all parameters with the exception of parameters 2,3, and 
4. 

Size parameters must be emphasized as being the main source of interactions; 
width (2) and length (3) are included in the five highest second-order elementary 
effects in the two presentations. For the annual heating load (Fig. I5d[) . their 
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own interaction (2-3) is the main one followed by their interaction with the 
set point temperature (4) and the ventilation rate (10). These results are fully 
consistent with the first-order analysis (Fig. I5b[) which pinpoints parameters 
2, 3, 4, 10 and 23 (insulation) as the major influences and highly scattered 
elementary effects. In the logarithmic transformation, insulation thickness is 
clearly the factor with the highest sigma values (highly scattered elementary 
effects) so it is not surprising to see interactions with size parameters among 
the main second-order interactions. High interactions with occupant heat gain 
(8) in effects (3; 8) and (2; 8) are more surprising. A possible interpretation 
could be that free heat gains have an almost additive influence on the space 
heating load as indicated by the fact that parameter 8 is the only one in the 
linear zone in the first-order analysis (Figs. I3"aland [5bl) . Dividing by the building 
volume introduces high interactions with size parameters while the logarithmic 
transformation introduces non-linearity for such additive phenomena: with the 
different transformations (Fig. I3bl I5cl and I3dl) input parameter 8 is no longer 
in the linear zone. No other factors are involved in the significant second-order 
interactions with occupants. Taking into account the size effect of occupants 
would suggest using occupants / 'm 3 (or occupants /m 2 of — floor — area) as an 
input parameter instead of just occupants in further work. 
Regarding the possible linearity of second order effects, no pair of parameters in 
Figures I5cl and [5dl are in the linear zone (a/u* < 0.1), which means that all pairs 
of parameters have non-linear effects or are combined with other parameters in 
the higher order. This can be compared to the first-order analysis for which 
only parameter 4 (set point temperature) falls in the almost linear zones (Fig. 
5aj) . Continuing the analysis of parameter 4, Figure [5cl emphasizes the strong 
interactions of this parameter with the size of the building (parameters 2 and 
3). These interactions remain important even if the size effect is reduced by 
considering the heating needs per cubic meter as the output. Parameter 23 
(insulation thickness of external walls) appears in all experiments (Figure 
with high variability in the first order as well as in the second order. The 
non-linear influence of the insulation thickness can explain this high variability. 



18 



In contrast, the first-order elementary effect of set point temperature (4) (non- 
monotonic for the annual heating needs) moves to the linear zone after logarith- 
mic transformation per cubic meter, with the highest influence. The standard 
deviation remains non-negligible if compared with other factors and a second- 
order interaction with size parameters appears even with the logarithm, but 
other interactions are limited. 

4-3.4- First order - small interval (B3) 

The same kind of analysis in Figure [5] can be found in Figure [5] where a small 
interval is considered. The main consequence of reducing the interval size of 
parameters (by a factor 10 for most of input parameters) is that almost all 
parameters are now in the linear zone for the first and second order. The set 
point temperature is again the most important parameter in this experiment 
with the highest average and standard deviation of the first-order elementary 
effects (all outputs in Fig. \§§. The importance of set point temperature is 
highlighted by the fact that its variation interval is kept rather wide in the 
experiment (reduction by a factor 4 only if compared to the large interval) . In 
the reverse, insulation thickness has now almost no influence on the output, due 
to a choice of rather well insulated wall (between 9 and 10 cm) for which the 
precise thickness is of little importance. 

4-3.5. Second order - small interval (B4) 

As first-order analysis results show almost a linear behavior for most of input pa- 
rameters, one could consider that second-order analysis could be of little interest. 
Actually, results can be considered as "second-order" magnitude : Second-order 
average elementary effects are smaller than 0.3% in the logarithmic output, e.g.. 
Moreover, they are close to the standard deviation of the first order analysis in 
the two presentations -kWh or Ln(kWh/m 3 )-, showing that scattering of first 
order EE are mainly caused by interactions between input parameters and not 
by quadratic or other non-linear influence. For the yearly energy output (kWh; 
Fig. I6d[) , the four factors dominating the first order analysis (2, 3, 4 and 10) 
are encountered again: the six main second-order EEs correspond to their six 
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pairs of double interactions, with interactions between size parameters (2, 3) 
and set-point (4) being the largest values. In the logarithmic output (Fig. I5c|) . 
the interactions between size parameters (2, 3) and occupant heat gains (8) 
analyzed in section 4.3.3 for the large interval are encountered again. Size pa- 
rameters are not involved in any significant interactions. But other interactions 
involving the set point temperature (4), the most influential parameter, are also 
identified with night- reduction set point (6) and occupant heat gains (8). For 
the first time in our analysis, input parameters related to a specific thermal zone 
of the building are shown as non negligible, with interaction between set point 
temperature in the whole building (4) and in this specific zone (5). Significant 
heat transfer between zones could explain this phenomenon which could become 
significant in situations where building characteristics are precisely known and 
important uncertainties could be related to diverse inhabitant behaviors in the 
various zones of the building. Standard deviations of second order elementary 
effects (in the logarithmic output) are always smaller than 0.02%, so third- or 
upper-order analysis will bring no useful information. 

5. Conclusions 

In this work, first- and second-order sensitivity analyzes have been combined. 
The Morris method and its extension have been applied to a building ther- 
mal simulation case study, using ESP-r to calculate the output. Among the 
results of this approach, the first order-analysis has demonstrated that, even 
if a precise ranking between the input parameters is not relevant, they can be 
split into different families in order to discuss their importance. This first order 
analysis helps to identify possible non-linearity or interactions of higher orders. 
The usefulness of considering various forms for the output function (kWh/year, 
kWh/year.m 3 , ln(kWh/year.m 3 )) has been explored, in particular to reduce the 
number of variables affected by these non-linearities or interactions. For in- 
stance, specific values per m 3 (or per m 2 ) reduce the correlation to the size- 
related parameters. Moreover, considering the logarithm of the output helps to 
identify the origin of some of the non-linearity. It is worth noting that investigat- 
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ing various model outputs does not require additional simulation runs: the same 
set of simulation trajectories is used and only complementary post-processing 
is needed. This remark can be extended to any transformed output calculated 
from model output and input variables. For those parameters still remaining in 
the area of high standard deviation, it has been demonstrated how the imple- 
mentation of the second-order sensitivity analysis can be helpful to sort variables 
and to specify their interaction in pairs. It has also been shown how the useful- 
ness of the upper-order analysis is amplified when combined with the different 
forms for the output. A new way of presenting results from the Morris method 
has been proposed to classify parameters (or couples of parameters in the case 
of second-order analysis) according to their sensitivity: linear- , monotonic- , 
almost monotonic or highly non-linear/interaction-of-higher-order). The value 
of sensitivity analysis using the elementary effects method has been clearly es- 
tablished. In any case, for a given building being simulated with a specific 
modeling tool, no general sensitivity can be derived: the results depend on the 
input parameters, with fixed values or varying values (and for the latter, their 
variation range). Thus, sensitivity analysis must be performed for each partic- 
ular situation, in relation to modeling goals, with a careful choice of variation 
intervals. Its conclusions are valid only for this particular situation. 
The choice of variation interval for each input parameter must be related to 
either a constrained range for decision variables or an uncertainty domain for 
exogenous variables. 
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Fig. 1. The apartment building case study. 

Fig. 2. Scatter plot of (<7,//i*) vs. (<Xj / abs (//*)) for all first-order analyzes 
performed (sets A1,B1,B2). 

Fig. 3. Estimated absolute average (/i*) and standard deviation (a) of the first- 
order elementary effects for different sets of outputs related to energy consump- 
tion (see Table [2] experiment Al). The numbers on the plot are the parameter 
indices and the lines represent the slope a/fi* at the values 0.1, 0.5 and 1. 
Fig. 4. Estimated absolute average (/!*) and standard deviation (a) of the 
first-order elementary effects in experiments for different sets of outputs related 
to power and thermal comfort (see Tabled experiment Al). The number in 
the plot are the parameter indices and the lines represent the slope a//j,* at the 
values 0.1, 0.5 and 1. 

Fig. 5. Estimated first- and second-order elementary effects for two outputs 
related to annual heating needs in kWh and in ln(kWh/year/m 3 ) with a large 
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interval between the parameters and r — 10. Both methodologies have the same 
number of parameters (Tabled experiments Bl and B2) The lines represent the 
slope a I [i* at the values 0.1, 0.5 and 1. 

Fig. 6. Estimated first- and second-order elementary effects for two outputs 
related to annual heating needs in kWh and in ln(kWh/year/m 3 ) with a small 
interval between the parameters and r — 10 (table [5]). The lines represent the 
slope a J ' \i* at the values 0.1, 0.5 and 1. 



Tables 



Year built (Base case) 1990 

Floor area (m 2 ) 3500 

Width (m) 14.00 

Length (m) 33.40 

Height (m) 18.00 

U-valuc external walls (W/m 2 K) 0.472 

U-valuc internal walls (W/m 2 K) 4.400 

U-valuc double-glazing (W/m 2 K) 2.811 

U-valuc basement floor (W/m 2 K) 0.887 

Glazing ratio (%) 30 

Number of occupants 70-80 

Table 1: Main characteristics of the apartment building test. 
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N Parameter 



Intervals 



1 (large) 


Experiment 


2 (small) 


Experiment 




50-150 


A 








50-150 


A,B1,B2 


90-100 


B3,B4 




50-150 


A,B1,B2 


90-100 


B3,B4 




17-24 


A,B1,B2 


20-22 


B3,B4 




17-24 


A,B1,B2 


20-22 


B3,B4 




0-8 


A,B1,B2 


0-1 


B3,B4 




0-8 


A 








1-8 


A,BLB2 


3-4 


B3,B4 




1-8 


A 








40-100 


A,B1,B2 


80-90 


B3,B4 




40-100 


A 








5-50 


A,BLB2 


45-50 


B3,B4 




5-50 


A 






CO 


5-50 


A 








5-50 


A 








20- 30 


A 








30-50 


A 








30-40 


A 








0.2-1 


A,B1,B2 


0.9-1 


B3,B4 




0.2-1 


A 








0.2-1 


A,BLB2 


0.2-1 


B3,B4 




0.5-1 


A 








5-100 


A,B1,B2 


90-100 


B3,B4 




0-180 


A,B1,B2 


0-10 


B3.B4 





1 Building size: correction for height [%] 

2 Building size: correction for width [%] 

3 Building size: correction for length [%] 

4 Set point temperature of all apartments except Apt.E42 [°C] 

5 Set point temperature of Apt. E42 [°C] 

6 Night-day set point temp. diff. affecting all apartments except Apt. E42 [°C] 

7 Night-day set point temperature of Apt. E42 [°C] 

8 Occupants affecting all apartments except Apt. E42 [occ./apt.] 

9 Occupants of Apartment. E42 [occ./apt.] 

10 Ventilation rate affecting all apartments except Apt. E42 [%] 

1 1 Ventilation rate of Apartment E42 [%] 

12 Glazing ratio A [%] 

13 Glazing ratio B [%] 

14 Glazing ratio C [%] 

15 Glazing ratio D [%] 

16 Ground reflectivity [%] 

17 Ground reflectivity in presence of snow (January-December) [%] 

18 View factor of ground [%] 

19 Climatic sensitivityxorrection for horizontal diffuse solar rad. [-] 

20 Climatic sensitivityxorrection for external dry bulb temp. [-] 

21 Climatic sensitivityxorrection for direct normal solar intensity [-] 

22 Climatic sensitivityxorrection for wind speed [-] 

23 Insulation thickness of external walls [mm] 

24 Building rotation [degrees] 



Table 2: List of parameters used in the Morris method first- and second-order experiments and the different intervals used in 
the analysis. 
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(a) Analysis of the elementary effects related to annual heating needs 
with a large interval in each parameter and r = 10. 
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(b) Analysis of the elementary effects related to the logarithm of 
annual heating needs with a large interval in each parameter and 

r = 10. 
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(c) Analysis of the elementary effects related to annual heating 
needs per cubic meter with a large interval in each parameter and 
r = 10. 
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(d) Analysis of the elementary effects related to the natural logarithm 
of annual heating needs per cubic meter with a large interval in each 
parameter and r = 10. 
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(a) Analysis of the elementary effects related to a comfort factor 
defined as the mean temperature in summer (July and August) with 
a large interval in each parameter and r = 10 



0.5 








23 / 


a/u*=1.Q 

ct/ji*=0.5 


¥ 










c/fi*=0.1 


^L 
























cB 0.4 








/ 


- 


g 








/ 




o 








/ 




Q. 








/ 




c 








/ 




e - 3 










. 2 


C 






/ 






O 






/ 


















CO 






/ 






> 0.2 






/ 




- 


0) 












T5 




.6 






- 3 ^.4-""""'" 


T5 




•24 








CO 




/ 






" 


"g 0.1 




/ /.8 


.10 


*$-'-""'" 




CO 












CO 

n 




J -19 
■21 









average p.* in Power[kW] 

(b) Analysis of the elementary effects related to power exceeded 
during 1000 hours/year. Each parameter has a large interval and 
the analysis has ten elementary effects per parameter (r = 10). 
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(a) Analysis of the first-order elementary effects related to annual 
heating needs with a large interval in each parameter and r = 10. 
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(b) Analysis of the first-order elementary effects related to annual 
heating needs with a large interval in each parameter and r = 10. 
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Figure 5 
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(a) Analysis of the first-order elementary effects related to annual 
heating needs with a small interval in each parameter and r = fO. 
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(b) Analysis of the first-order elementary effects related to annual 
heating needs with a small interval in each parameter and r = fO. 
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(c) Analysis of the second-order elementary effects related to annual 
heating needs with a small interval in each parameter and r = fO. 
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(d) Analysis of the second-order elementary effects related to an- 
nual heating needs with a small interval in each parameter and 
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